1. Field of the Invention
The present invention relates generally to photocurrent estimation and, more particularly, to photocurrent estimation from multiple captures for simultaneously improving signal-to-noise ratio (SNR) and dynamic range in complementary metal oxide semiconductor (CMOS) image sensor systems.
2. Description of the Related Art
An image sensor converts an optical image focused on the sensor into electrical signals. The electrical signals represent the intensity of the image. It is important that the image sensor be usable under a variety of lighting conditions because the wider the variety of lighting conditions under which the image sensor is usable the better the image quality. Consequently, the quality of an imaging system is commonly measured by the image sensor's dynamic range and its ability to mask noises, i.e., its SNR, under low light conditions.
The dynamic range of an image sensor measures how wide a range of lighting the sensor can accurately capture. For example, a scene including both a tree lit by bright sunlight and a person standing under the tree has a high dynamic range. This high dynamic range makes it difficult for the image sensor, such as one used in a camera, a video recorder, or a security monitor, to capture details of both the brightly lit tree and the person standing in the tree's shadow.
The wider the dynamic range of the image sensor, the more details can be shown under extreme conditions and thus the more versatile the associated imaging system becomes. For example, with a wide dynamic range, a novice or an inexperienced imaging system user can take great pictures or videos without worrying much about lighting conditions and/or corresponding settings. Even an advanced or experienced user is likely to enjoy the convenience and advantages of a wide dynamic range imaging system.
The SNR of an image sensor measures the ratio between the signal and its associated noise. An image sensor with low SNR will have an objectionable amount of static-like or grain-like noise appearing in the captured image, while an image sensor with high SNR can be used under low lighting conditions, such as in a room lit only by candlelight. Clearly, an ideal imaging system would desirably have a high SNR as well as a wide dynamic range.
Another desirable feature of an ideal imaging system is the prevention/elimination of motion blur, allowing a quality image to be captured from a moving subject. In the case of a camera, motion blur may be controlled by shutter speed. However, the effectiveness of this technique is dependent upon illumination level. As such, a highly desirable imaging system would be one that is capable of automatically adjusting itself in order to compensate a wide range of illumination levels and one that produces noise free and motion blur free high quality images even when the subject may be moving fast and/or unexpectedly. Virtually all imaging systems and image sensing devices can benefit from utilizing image sensors that offer wider dynamic range, higher SNR, and motion blur free images.
Most of today's imaging systems and image sensing devices such as video and digital cameras use charge-coupled device (CCD) image sensors. In CCD image sensors, the electric charge collected by the photo detector array during exposure time is serially shifted out of the sensor chip, resulting in slow readout speed and high power consumption. Furthermore, since CCDs are fabricated in a non-standard technology, other analog and digital camera functions such as A/D conversion, image processing and compression, control and storage cannot be integrated with the sensor on the same chip and must be implemented using several other chips. Such implementation can be quite expensive because of the specialized processing involved for CCDs.
CCD image sensors are well known in the art and thus are not described herein. An exemplary teaching can be found in U.S. Pat. No. 5,272,535, which is incorporated herein by reference, titled “Image Sensor with Exposure Control, Selectable Interlaced, Pseudo Interlaced or Non-Interlaced Readout and Video Compression”, issued to Elabd of Sunnyvale, Calif., and assigned to Loral Fairchild Corporation, Syosset, N.Y., December 1993.
The CMOS technology provides the possibility of integrating image sensing and digital signal processing on the same chip, resulting faster, smaller, less expensive, and lower power image sensing devices. The advantages of CMOS image sensors over CCD image sensors are well known. An exemplary teaching, which is incorporated herein by reference, can be found in Wong's “Technology and Device Scaling Considerations for CMOS Imagers”.
Recently developed CMOS image sensors are read out non-destructively and in a manner similar to a digital memory and can thus be operated at very high frame rates. Several high speed CMOS Active Pixel Sensors have been recently reported. In “A High Speed, 500 Frames/s, 1024×1024 CMOS Active Pixel Sensor”, Krymski et al. describe a 1024×1024 CMOS image sensor that achieves 500 frames per a second. Stevanovic et al. describe in “A CMOS Image Sensor for High Speed Imaging” a 256×256 sensor achieving 1000 frames per a second. In “A 10,000 Frames/s 10.18 μm CMOS Digital Pixel Sensor with Pixel-Level Memory”, Kleinfelder et al. describe a 352×288 CMOS Digital Pixel Sensor achieving 10,000 frames per a second.
Unlike CCD image sensors, a CMOS image sensor can be integrated with other camera functions on the same chip ultimately leading to a single-chip digital camera with very small size, low power consumption and additional functionality. The integration of processing and image capture coupled with high frame rate capability of CMOS image sensors enable efficient implementations of many still and standard video imaging applications. A drawback, however, is that CMOS image sensors generally suffer from lower dynamic range and SNR than CCDs due to their high readout noise and non-uniformity.
It has been proposed to enhance dynamic range via multiple image captures (multiple sampling). The idea is to capture several images at different times within the normal exposure time—shorter exposure time images capture the brighter areas of the scene while longer exposure time images capture the darker areas of the scene. (In video imaging applications, the video frame rate sets an upper bound on the exposure time. In digital still photography, the maximum exposure time is limited by possible motion of the digital camera, possible motion of the subject, and saturation of the sensors.) A high dynamic range image is then synthesized from the multiple captures by appropriately scaling each pixel's last sample before saturation.
In “Comparative Analysis of SNR for Image Sensors with Enhanced Dynamic Range”, which is hereby incorporated herein by reference, Yang et al. show that this scheme achieves higher SNR than other dynamic range enhancement implementations. However, this scheme does not take full advantage of the captured images. Since readout noise is not reduced, dynamic range is only extended at the high illumination end. Furthermore, according to this prior art scheme, although dynamic range can be extended at the low illumination end by increasing exposure time, such increase in exposure time results in unacceptable blur due to motion or change of illumination.